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The purpose of this implementation support plan is to provide 
detailed information to state and local Triple P coordinators, v2 
funders, policymakers, and technical assistance providers about 


the core principles, partner roles, coordinated activities, and 
outcomes for the provision of active implementation support to 
NC counties scaling the Triple P system of interventions. 


Background 


Evidence-based prevention and wellbeing programs offer a great deal of promise to support the 
health and wellbeing of North Carolina children, youth, families, and communities. In fact, many 
funders and service providers in North Carolina are shifting towards models that have 
demonstrated positive impact through rigorous evaluations. However, implementing and scaling- 
up these innovations can be a challenge in the context of business as usual. Despite best 
intentions, longstanding, complex service systems have a tendency to pull innovation back to 
past practice. This challenge can prevent evidence-based strategies from achieving expected 
outcomes, including in North Carolina. 


The North Carolina Implementation Capacity for Triple P (NCIC-TP) project aims to develop 
methods, materials, and opportunities to support North Carolina counties to successfully and 
sustainably scale the Triple P — Positive Parenting Program (Triple P) system of interventions so 
that population-level benefits are realized for local children, families, and communities. To 
address typical challenges related to implementation and scale-up, NCIC-TP leverages: 


(1) Current research and best practices from implementation science, 


(2) Mixed-methods evaluation findings from the North Carolina Triple P Implementation 
Evaluation (TPIE), and 


(3) Partnerships with statewide stakeholders involved in scaling-up the Triple P system. 


Implementation Science 

Current research and best practices from implementation science provide the backbone for NCIC- 
TP. Research and applied learning from efforts to successfully implement and scale evidence- 
based programs have been amassing over the past two decades.?* Among ready organizations 
and systems, developing and sustaining local capacity around core implementation processes 
have emerged as essential parts of success and sustainability.*° Although a number of 
implementation science frameworks are now available to make sense of key implementation 
concepts, the most promising approaches to implementation and scale-up give strong attention 
to three key features of local implementation capacity: 
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(1) Linking local leadership and implementation teams within (e.g., individual service 
agencies) and across (e.g., community coalitions led by local backbone organizations, 
statewide intermediary organizations, and state service agencies to support 
implementation) levels of community service systems;**°2° 


(2) Best practices for practitioners’ professional development (i.e., selection, training, 
coaching) to deliver programs as intended and with expected benefits for children and 
families;21"34 and 


(3) Quality and outcome monitoring for systems or organizational improvement and program 
optimization.*273>-44 


For communitywide prevention and wellbeing efforts, developing media and networking 
strategies to mobilize knowledge and behavior change also appears to be important for achieving 
population-level outcomes.'*4349 NCIC-TP makes systematic efforts to blend leading-edge 
implementation science and best practices into all evaluation and implementation support 
activities. 


North Carolina Triple P Implementation Evaluation (TPIE) 

To bring the science of implementation closer to the ground-level in North Carolina, NCIC-TP was 
also founded on two implementation-science-based evaluations of Triple P in North Carolina: 
TPIE and TPIE-Qualitative. From January 2014 through December 2015, TPIE evaluators examined 
the implementation and scale-up of the Triple P system in Cabarrus and Mecklenburg counties. 
Specifically, the purpose of TPIE was to evaluate capacity and infrastructure for the active 
implementation of Triple P to inform the planning process for impact and sustainability. In late 
winter and early spring 2016, the TPIE team added a qualitative evaluation component (TPIE- 
Qualitative) to better understand the findings from the initial implementation evaluation and 
further improve the planning process for Triple P impact and sustainability. Although highlights 
of evaluation results are touched upon in this section, detailed evaluation backgrounds, 
evaluation findings, and lists of evaluators’ recommendations are available in the TPIE Final 
Report°? and the TPIE-Qualitative Report, both of which are located on the NCIC-TP website at 
http://ncic.fpg.unc.edu/lessons-learned. 


TPIE results°° highlighted several strengths of local Triple P implementation capacity in Cabarrus 
and Mecklenburg counties during the evaluation period, including the capacity of county Triple 
P leadership teams, the capacity of Cabarrus County’s Implementation Team, counties’ Triple P 
practitioner recruitment and selection processes, local Triple P training processes, and county- 
level Triple P decision-support data systems and quality improvement processes. In addition to 
these strengths, four areas of implementation capacity needed particular attention and further 
development: agency implementation team structures; infrastructure to support Triple P 
practitioners’ ongoing coaching following accreditation; Triple P fidelity assessment practices; 
and infrastructure for using Triple P data and feedback about implementation barriers and 
facilitators for agency Triple P quality improvement. 
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Four risk factors for service agencies’ discontinuation of Triple P implementation were also 
identified during TPIE, including having: 


(1) Only one Triple P practitioner within the service agency (this does not pertain to 
private/independent Triple P practitioners); 


(2) Less developed agency Triple P leadership and implementation team structures; 


(3) A less hospitable agency implementation climate for Triple P, which may be indicative 
of lower agency leadership and management support for Triple P;>? and 


(4) Less formalized or documented agency Triple P sustainability plans. 


At the county-level, TPIE results also suggested that the successful scale-up of Triple P may be 
more challenging if the county implementation team has less capacity (particularly in terms of 
formally allocated time and effort for team members), the county doesn’t have adequate 
financial resources to support local Triple P scale-up, and if the county prevention system and 
population are larger or are more complex. 


TPIE-Qualitative results?! reinforced many of these initial TPIE findings and added a handful of 
additional important points about Triple P scale-up in NC counties. These included: the need for 
more active implementation support to counties scaling, and agencies implementing, Triple P; 
the need for robust exploration and readiness processes at each level of the statewide Triple P 
system before embarking upon local Triple P adoption or installing new features of Triple P 
implementation; the benefits of using a coalition approach to locally scaling Triple P in NC 
counties and ensuring a statewide learning collaborative for county Triple P coordinators; and 
the need for more actively and purposefully involving community members in the Triple P 
implementation infrastructure. 


Co-Creation Partnerships 

Finally, NCIC-TP was developed around a co-creation model of applying implementation science 
within local contexts. While the science of implementation provides meaningful direction, the 
utilization of strategies from implementation science, and the development of local 
implementation infrastructure, requires co-creation from five partners:°? 


(1) Service agency leadership and staff from implementing sites; 
(2) State/local funders and policymakers; 


(3) Intermediary and purveyor organizations that provide implementation and program 
support (i.e., implementation technical assistance providers, Triple P America); 


(4) Active and involved community members (e.g., community parents and youth being 
served); and 


(5) Intervention developers and prevention scientists. 


The successful and sustainable scale-up of Triple P in North Carolina and the realization of 
population-level prevention and wellbeing benefits will necessitate collaborative partnerships 
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among all five co-creation partners. NCIC-TP responds to opportunities for co-creation and 
humbly accepts that the work of implementing and scaling Triple P cannot be accomplished by 
one or two of these partners alone. 


Implementation Support for NC Counties Scaling Triple P 


The purpose of this implementation support plan is to detail core principles, processes, features, 
partner roles, and intended outcomes for the provision of active implementation support to 
North Carolina counties scaling the Triple P system of interventions. In doing so, the plan takes a 
customizable and adaptive approach>*°° to supporting implementation processes rather than 
prescribing a series of specific steps and procedures. 


Objectives of Implementation Support 
Active implementation support provided to NC counties scaling the Triple P system of 
interventions seeks to contribute to several objectives. 


(1) Strengthening a multi-level system of implementation and program support from state 
to counties to agencies to practitioners to families. 


(2) Organizing and aligning communitywide implementation capacity. This includes 
ensuring adequate implementation capacity within lead or backbone agencies and service 
agencies participating in local Triple P coalitions. 


(3) Supporting implementation performance across lead/backbone agencies and local Triple 
P service agencies. 


(4) Locally scaling the Triple P system to respond to identified community needs, 
characteristics, and readiness. 


(5) Supporting practitioners’ delivery of Triple P interventions as intended and in response 
to parents’ needs and preferences. 


(6) Increasing the probability that intended prevention and wellbeing outcomes will be 
achieved at scale. 


(7) Sustaining Triple P implementation and program performance. 


Principles of Implementation Support 
Across all partners, the provision of implementation support for NC counties scaling the Triple P 
system of interventions benefits from being guided by several principles.* 


(1) Change requires proactive support: Like other efforts to change individual and group 
behavior, implementing and scaling evidence-based prevention strategies requires 
intentional and focused support. Proactive implementation support anticipates needs 
and incorporates strategic approaches to bring new knowledge, skills, and opportunities 
for recipients to apply and test new learning — with reinforcement and supportive 
feedback — in their own systems environments. Such learning and support is often 
necessary at individual, team, organizational, and system levels. 
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(2) Use of implementation science and best practices: As we ask community leaders and 
local practitioners to be guided by the science of prevention, so too must co-creation 
partners involved in supporting implementation and scale-up be guided by the science of 
innovation implementation. A range of frameworks and tools are now available from 
implementation science to support the introduction of key concepts and strategies for 
effective implementation. Furthermore, these frameworks and tools can inform 
exchanges of ideas with local stakeholders to enable local application and sustainability. 
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Co-creation: The development of local implementation infrastructure is becoming 
recognized as a process of co-creation.°?°° Within the co-creation framework, five 
partners contribute to successful and sustainable implementation and scale-up: 


a. Service agency leadership and staff from implementing sites; 
b. State/local funders and policymakers; 


c. Intermediary and purveyor organizations that provide implementation and 
program support; 


d. Active and involved community members (e.g., community parents and youth 
being served); and 


e. Intervention developers and prevention scientists. 


Support for active implementation and scale-up becomes stronger as collaborations and 
contributions among these five partners increase. 
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Contextualized and responsive support: While the science of implementation provides 
meaningful grounding for any implementation effort, to increase chances for success and 
sustainability, efforts to implement and scale-up prevention programs must be optimized 
within local contexts.1”°”7° Implementation strategies need to be considered and tailored 
according to key features of local prevention systems, such as size, history, resources, 
culture, population density, and political and social complexities. Furthermore, ongoing 
implementation support needs to be responsive to local progress, setbacks, feedback, 
and key events. 
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Adaptive leadership: Implementation and scale-up are adaptive processes, not technical 
processes.’ Implementation support partners must develop an appreciation for, and 
comfort with, the diverse perspectives held within local systems environments and begin 
to recognize these as clues to the presence of adaptive challenges embedded within the 
system and its people. Heifetz and Laurie’? put forward six principles of adaptive 
leadership that can be used to manage adaptive challenges: 


a. Get on the balcony: step back from daily system operations to see larger patterns 
of individual and collective behavior and local history that may be either 
facilitating or hindering the systems’ willingness or ability to change. 
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b. Identify the adaptive challenge: take time to clearly define adaptive challenges. 
Definitions should take into account an understanding of local people, 
organizational and community history, larger system pressures, and identified 
sources of conflict. 


c. Regulate distress: create a functional balance of system stress by using conflict as 
an opportunity for learning and creativity, sequencing and pacing work, and 
preventing stakeholders from feeling overwhelmed by change. 


d. Maintain disciplined attention: maintain focus on tough questions and prevent the 
avoidance of adaptive work recognized by sliding back into familiar routines or 
engaging peripheral issues or topics. 


e. Give the work back to people: build the collective problem-solving confidence of 
system stakeholders rather than provide expert solutions or let the burden of 
adaptive work fall on the few identified vocal leaders. 


f. Protect voices of leadership from below: ensure that the experiences and ideas of 
those often marginalized in change initiatives, including front line staff and 
community members, are voiced and play an equal role in generating solutions so 
that they will be the most successful and sustainable. 


(6) Data-driven progress monitoring and improvement: As advocates for the translation of 
evidence into practice, implementation support partners collect and use data to identify 
local needs and plan responsive support strategies, monitor the progress and outcomes 
of local implementation efforts, monitor the effectiveness of their own support, and make 
quality improvements based on data over time. 
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Local ownership of progress: Implementation support partners should promote local 
systems’ ownership of implementation processes and successes. Although external 
partners can be seen as instrumental to increasing implementation resources and 
abilities, ongoing success in implementation and scale-up should not be perceived to be 
due to, or dependent on, external support partners. This principle can be demonstrated 
by continually promoting collective-efficacy within community prevention systems. 


Implementation Logic Model 


Drawing heavily on Chinman and colleagues’ implementation TA logic model,’? a logic model that 
describes the relationships between key intermediate and long-term outcomes of active 
implementation support is provided in Figure 1. This logic model of implementation support is 
rather comprehensive and not meant to suggest a prescribed process for scaling Triple P. 
Rather, within certain limits, local communities and system partners might customize their level 
of use of this logic model. For example, local communities may choose to monitor only certain 
population-level or implementation outcomes articulated in this logic model. Likewise, co- 
creation partner roles may vary in intensity and function according to local community context. 
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It is recommended, however, that all communities establish collaborative relationships with the 
noted co-creation partners, as feasible. Also, attending to each component of local 
implementation capacity and performance articulated in the logic model may be essential to 
realizing the full impact of Triple P on population-level outcomes. 


Population-level Outcomes 

As a part of the North Carolina Triple P Statewide Evaluation, system partners have agreed to 
monitor three population-level outcome variables that have demonstrated responsivity in prior 
research on the countywide scale-up of the Triple P system in the Southeastern United States:’* 


(1) substantiated child abuse and neglect, 
(2) out-of-home foster care placements, and 


(3) child injuries treated in a hospital. 


In addition to these recognized statewide evaluation outcome variables, local Triple P coalitions 
may have interest and resources to monitor other child, family, and community outcomes that 
have demonstrated responsivity to Triple P use. Triple P, both through individual interventions 
and the aggregate system, has demonstrated positive child and family outcomes across a number 
of research and evaluation trials globally. Local Triple P coalitions may benefit from examining 
the full Triple P evidence-base, available at https://www.pfsc.ug.edu.au/research/evidence/. 
Readers can query the Triple P evidence-base according to individual Triple P interventions as 
well as key topics and outcome variables. 


NCIC-TP promotes the idea that intervention outcomes, whether individual or population-level, 
can be optimized in local context.2° We hope that stakeholders involved in county Triple P rollouts 
will take advantage of this perspective and strive to move beyond the level of outcomes 
established in prior Triple P research for the benefit of local communities. 


Triple P System Optimization 

Reviews of the research literature have made clear that implementation quality impacts the 
realization of outcomes when evidence-based prevention and wellbeing programs are used in 
the real world.”? Perhaps the most recognized feature of implementation quality is fidelity to the 
intended delivery of adopted programs. However, several other implementation outcomes may 
also be important, particularly as related to achieving favorable service and client outcomes at 
scale. For example, Proctor and colleagues’® offer eight core implementation outcomes: 
acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability. 
The NCIC-TP implementation logic model adapts and incorporates essential features of these 
eight implementation outcomes, and includes other implementation outcomes that may be of 
particular interest given Triple P’s model, history, and ongoing aims in NC counties: 


(1) Accessibility. Progressing beyond Proctor and colleagues’ simpler adoption outcome, 
accessibility is defined as the degree to which local families can access parenting and 
family support in accordance with the level of support they need or prefer. 
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Figure 1. Logic model for supporting the implementation and scale-up of the Triple P system of interventions in North Carolina to 
achieve population-level outcomes. 


11 


(2) System Alignment. Not represented in Proctor and colleagues’ original list but important 
for any system of interventions, system alignment is defined as the degree to which local 
service agencies or individual interventions work in concert towards collective wellbeing 
goals rather than in silos or fragmentation. 


(3) Feasibility. As defined by Proctor and colleagues (p. 69), feasibility is the extent to which 
a new treatment, or an innovation, can be successfully used or carried out within a given 
agency or setting. Feasibility has a lot to do with whether or not the local setting of care 
has the necessary financial, human, and implementation resources to support delivery 
of the intervention as intended. 


(4) Appropriateness. As defined by Proctor and colleagues (p. 69), appropriateness is the 
perceived fit, relevance, or compatibility of the innovation or evidence-based practice 
for a given practice setting, practitioner, or consumer; and/or perceived fit of the 
innovation to address a particular issue or problem. 


(5) Fidelity. As defined by Proctor and colleagues (p. 69), fidelity is the degree to which an 
intervention was implemented as it was prescribed in the original protocol or as it was 
intended by the program developers. Four dimensions of program fidelity relevant to 
Triple P in NC are detailed by Dane and Schneider’’ and later reinforced by Mihalic,’® (p. 
83) relative to prevention programs: 


a. Adherence refers to whether the intervention is being delivered as it was designed 
or written (i.e., with all core components being delivered to the appropriate 
population; staff trained appropriately; using the right protocols, techniques, and 
materials; and in the locations or contexts prescribed). 


b. Quality of program delivery is the manner in which a practitioner delivers a 
program (e.g., skill in using the techniques or methods prescribed by the program, 
enthusiasm, preparedness, and attitude). 


c. Caregiver engagement is the extent to which participants are engaged by and 
involved in the activities and content of the program. 


d. Dosage may include any of the following: the number of sessions implemented, 
length of each session, or the frequency with which program techniques were 
implemented. 


Dane and Schneider and, separately, Mihalic also discuss the fidelity dimension program 
differentiation, which may be more important within controlled research settings than 
Triple P scale-up in NC counties. 


(6) Acceptability: As defined by Proctor and colleagues (p. 67), acceptability is the 
perception among implementation stakeholders (e.g., families) that a given practice or 
program is agreeable, palatable, or satisfactory, as delivered. 
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(7) Reach: Proctor and colleagues use a synonymous term, penetration, which is defined as 
the integration of a practice within a service setting and its subsystems (p. 70). Reach 
might be measured by (a) the number of people who receive an intervention compared 
to those who are eligible to receive the intervention, or (b) the number of practitioners 
(actively) delivering the intervention compared to the number trained in or expected to 
deliver the intervention. TPIE results and experience from Triple P stakeholders in NC 
suggest that there has been a significant discrepancy between the number of 
practitioners trained in Triple P and those who remain actively delivering Triple P 
interventions to local families. 


(8) Cost: As defined by Proctor and colleagues (p. 67), cost is related to the cost impact of 
an implementation effort. Proctor and colleagues note three cost components may be 
of interest: 


a. costs of delivering the intervention, 


b. costs of the implementation strategies that will be used to support the 
intervention, and 


c. cost variability associated with the local service delivery setting. 


An additional variable related to cost, return on investment, has received increasing 
interest and attention relative to the implementation and scale-up of evidence-based 
practices’9, 


(9) Sustainability: As defined by Proctor and colleagues (p. 70), sustainability is the extent 
to which a newly implemented intervention is maintained or institutionalized within a 
service setting’s ongoing, stable operations. 


System stakeholders involved in different levels of community Triple P rollouts across NC (e.g., 
state, county, agency, and practitioner) may have varied interest across these nine 
implementation outcomes. While stakeholders may want to review these alternatives and 
determine which mix may be of most interest and usability at their system level, NCIC-TP strongly 
recommends that program fidelity is monitored by every system level. Fidelity has demonstrated 
particular importance in relation to the replication of evidence-based program outcomes in real 
world implementation.’”’8 In addition, by choosing from and attending to other implementation 
outcomes, such as acceptability and appropriateness, we believe that system stakeholders at any 
level can monitor implementation in accordance with Triple P’s stated philosophy of “fidelity and 
flexibility.” Monitoring variables like acceptability and appropriateness can ensure that 
interventions core components are reaching local families in a way that is responsive to family 
needs and preferences. 


Finally, NCIC-TP promotes the idea that implementation outcomes, like population-level 
outcomes, can be optimized in local context.?° Hence, we refer to this section of the logic model 


NCIC-TP 


North Carolina Implementation 
Capacity for Triple P 


13 


as Triple P System Optimization to reflect the perspective of continuous quality improvement 
within local contexts. 


Implementation Performance 

Implementation outcomes are influenced by the level of local implementation capacity and 
performance.’? Chinman and colleagues define implementation performance as “the level of 
quality at which [key implementation support practices] are carried out” (p. 3). The NCIC-TP 
implementation logic model details five core implementation support practices: 


(1) Leading and supporting Triple P implementation and scale-up, including identifying and 
addressing implementation barriers and spreading successes; 


(2) Developing competent and confident Triple P practitioners who can deliver Triple P with 
fidelity and flexibility; 


(3) Gathering, analyzing, and reporting to the right people at the right times program and 
implementation data related to Triple P delivery; 


(4) System-wide learning and continuous quality improvement of Triple P implementation, 
delivery, and outcomes; and 


(5) Mobilizing knowledge and behavior change across communities beyond that created by 
direct service interventions. 


We believe these performance indicators provide congruity with core components of local 
implementation capacity as described in the next section. These performance indicators are high 
level and may be further broken down into more specific performance behaviors. For example, 
leading and supporting Triple P implementation may involve executive leaders’ ongoing 
demonstration of commitment to Triple P implementation (i.e., “implementation climate”) and 
aligning community prevention strategies under common approaches and outcomes of 
implementation.®®° Likewise, developing competent and confidence practitioners may involve 
high quality practitioner recruitment and selection, training, and coaching practices.227,8° 


Local Implementation Capacity 

As discussed earlier, NCIC-TP takes a perspective that the most promising approaches to 
implementation and scale-up give strong attention to three key features of local implementation 
capacity: 


(1) Linking local leadership and implementation teams within (e.g., individual service 
agencies) and across (e.g., community coalitions led by local backbone organizations, 
statewide intermediary organizations, and state service agencies to support 
implementation) levels of community service systems;7**?° 


(2) Best practices for practitioners’ professional development (i.e., selection, training, 
coaching) to deliver programs as intended and with expected benefits for children and 
families;2+34 and 
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(3) Quality and outcome measurement and monitoring for systems or organizational 
improvement and program optimization.4273>-44 
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Figure 2. Cascading implementation support for Triple P across NC county prevention system 
levels. 


For communitywide prevention and wellbeing efforts, developing media and networking 
strategies to mobilize knowledge and behavior change also appears to be important for achieving 
population-level outcomes.'44349 


NCIC-TP recognizes the existing multi-level system of support for the scale-up of Triple P that has 
already started to develop across North Carolina. Such cascading models of implementation 
support may provide an effective way to promote meaningful capacity and roles within each level 
of a statewide system and support overall success.’*?’ Figure 2 presents the state’s multi-level 
system of Triple P support, within which key features of implementation capacity and 
performance are to be embedded. While feedback loops are presented between single levels of 
the system, it is acknowledged that feedback from and to each level of the system is likely 
happening and is important for quality improvement and increasing the likelihood of success. 


Co-Creation Partner Support 
Co-creation partners working collaboratively to support county Triple P rollouts across NC 
contribute to the development of local implementation capacity and performance.>*** Key 
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features of co-creation partner roles follow below yet are not intended to be comprehensive. 
Individual co-creation partners may serve a variety of unique and shared functions and the nature 
and intensity of partner roles may shift and change over time as implementation and scale-up 
progresses. 


Local agency leadership & staff 

Local leadership and staff within Triple P coalition lead agencies and service agencies have a key 
role in generating initial readiness for implementation and scale-up of Triple P. Local readiness 
for implementation can be understood as a combination of stakeholders’ commitment to the 
change process and their collective belief that they can make the changes that will be required.®* 
Several factors may influence local readiness for change, such as the value placed on making the 
change, task demands that may be involved, resource availability, and relevant situational 
factors. 


Beyond generating initial readiness for Triple P implementation and scaling, local leadership and 
staff must continuously ensure several other leadership and coordination functions for actively 
implementing and scaling Triple P.2 Those with executive leadership of implementing and scaling 
Triple P — whether within community Triple P coalition lead agencies or individual service 
agencies — may support success and sustainability by demonstrating ongoing commitment to 
the change process and change partnerships, and by creating and nurturing appropriate 
opportunities for change within local organizations and systems.? 


Those leading the development of the community Triple P coalition must also ensure that Triple 
P and related family service initiatives are well aligned and usable by practitioners, coalition 
policies and agency practices facilitate delivery of Triple P interventions as intended, and system 
changes and successes are well communicated across stakeholders and community members.® 


Finally, those with day-to-day management responsibilities for Triple P program implementation 
and scaling — whether within community Triple P coalition lead agencies or individual service 
agencies — support success and sustainability by ensuring ongoing buy-in and readiness for 
stage-based scale-up of Triple P in the community; organizing, aligning, and sustaining the 
necessary infrastructure to support Triple P implementation within the community; and by 
actively using data and other information for quality improvement of Triple P implementation.® 


These leadership and coordination functions can be institutionalized within leadership and 
implementation team structures at the agency and community Triple P coalition levels. 
Furthermore, they may be essential to the development and _ sustainability of local 
implementation capacity and performance more broadly,®1?” and may support hospitable 
agency and coalition climates for implementing and scaling Triple P.°2 


State/local funders & policymakers 
State and local funders and policymakers have an important role in creating a nurturing systems 
environment for county Triple P rollouts. In particular, key functions include: 
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(1) Ensuring the availability of adequate financial resources to develop necessary 
implementation capacity and support the delivery of Triple P, 


(2) Ensuring adequate time and space to reasonably expect implementation and scale-up to 
translate into population-level outcomes, and 


(3) Setting expectations and resources for quality and outcome monitoring of Triple P 
across all levels of the Triple P system. 


When funding comes from state agencies, they may also play a supportive role in coordinating 
state-level learning collaboratives, statewide implementation support teams, and statewide 
intervention components (e.g., media-based intervention components). 


Triple P America 

Triple P America is the U.S.-based purveyor of Triple P training, materials, and implementation 
support. Recently, Triple P International published the Triple P Implementation Framework 
(TPIF), which details their role in supporting Triple P implementation and sustainability.°? TPIF 
details five phases of activities between Triple P America and local service systems adopting 
Triple P interventions. 


(1) Engagement: Initial interactions with community stakeholders to explore if Triple P is a 
good fit for their goals and community needs. 


(2) Commitment and Contracting: Confirmation of the scope of Triple P implementation and 
facilitation of written agreements for training, resources, and support. 


(3) Implementation Planning: Collaboration on creation of an implementation plan, 
including plans for communications, training and accreditation, service delivery, quality 
assurance, and evaluation. 


(4) Training and Accreditation: Delivery of standardized training and accreditation process 
for practitioners. 


(5) Implementation and Maintenance: Engagement in feedback cycles with community 
stakeholders around service delivery, quality improvement, ongoing development, and 
sustainability mechanisms. 


Across these five phases, Triple P America helps to support practitioner professional 
development to deliver Triple P interventions as intended, assure quality, enable outcome 
monitoring, and contribute to the development of local implementation capacity needed to 
support and improve local Triple P implementation. 


Intermediary organizations 

Intermediary organizations differ from program purveyors in that they support the dissemination 
and implementation of more than one evidence-based program or practice and, as such, have a 
more expanded role than program purveyors.*4 As defined by Mettrick and colleagues® (p. 3), 
an intermediary organization: 
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“Supports service array development through implementation technical assistance, 
creative financing options, training, coaching, education, continuous quality 
improvement monitoring, and outcomes evaluation. 


[An intermediary organization] connects providers, state agencies, local jurisdictions, and 
purveyors to ensure that effective implementation leads to improved outcomes and 
builds on existing systems reform efforts.” 


Because intermediary organizations are often more regionally located to implementation sites 
compared to the national or international presence of program purveyors, they are able to serve 
several unique functions. Through their recent Center of Excellence Learning Community funded 
by the Annie E. Casey Foundation, Mettrick and colleagues® detail five core functions for 
intermediary organizations: 


(1) Implementation support for evidence-based programs; 

(2) Research, evaluation, and data linking capacity; 

(3) Partnership engagement and collaboration; 

(4) Workforce development activities (including training and coaching); and 


(5) Policy and finance expertise. 


Intermediary organizations do not replicate the role of state agencies or program purveyors, 
rather they work in concert with state agencies, funders, and program purveyors to support the 
achievement of common goals. Where functions or activities overlap among any co-creation 
partners, developing clear agreements about roles and how to support synergistic, rather than 
contradictory work patterns, becomes essential. 


The NCIC-TP team is working to identify and build the capacity of an intermediary organization 
or partnership of common organizations to support Triple P system implementation in North 
Carolina. 


Local community members 

Local community members, including the children, youth, and families receiving services, play 
essential roles in the successful and sustainable implementation of evidence-based 
interventions, particularly at scale. Respondents in the TPIE-Qualitative evaluation identified that 
local community members were particularly helpful by:°4 


(1) Providing feedback and supporting continuous quality improvement of Triple P delivery 
at agency, county, and state levels; 


(2) Catalyzing Triple P engagement within their communities by word-of-mouth advertising, 
sharing positive experiences, and transferring learning and parenting skills to other 
community parents and stakeholders; 
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(3) Championing Triple P with local, county and statewide stakeholders; and by 


(4) Fully participating in Triple P implementation structures, such as decision-making bodies 
that select which Triple P programs to adopt locally. 


In addition, Boothroyd and colleagues®® detail five functions that active, involved partnerships 
between local service systems and community members can support during implementation and 
scale-up: 


(1) Listening to learn about and begin to address historical trauma (historical maltreatment 
of families in key communities identified by cultural factors such as race or income level), 
mistrust of agencies and systems, and other long-standing and institutional barriers to 
safety, health, and wellbeing; 


(2) Working with community members to identify system barriers to improved outcomes for 
children and families and implement action plans to address those barriers; 


(3) Collaborating with community members to establish culturally relevant supports and 
services to meet the underlying needs of children and families; 


(4) Meaningfully involving community members in practitioner professional development 
activities and community design teams for effective, sustained implementation; and 


(5) Ensuring partnership meetings, forums, and feedback loops are sustained so that 
community members are continuously connected to and help guide ongoing practice and 
system changes. 


Participants in TPIE-Qualitative suggested that, overall, there is a need for more actively and 
purposefully involving community members in local Triple P implementation activities and 
decision-making.>* 


Triple P developers & researchers 

Finally, Triple P developers and researchers have both proactive and reactive roles relative to the 
implementation and scale-up of Triple P. Proactively, Triple P developers need to ensure that 
Triple P programs and strategies are usable within community prevention systems.!22187.8 
Interventions that meet usability criteria are regarded as teachable, learnable, doable, 
repeatable, and assessable in practice.®”** Triple P researchers also have a key role to ensure that 
Triple P programs and media strategies are and remain evidence-based. This was one of the 
most widely identified roles of Triple P researchers during the TPIE-Qualitative evaluation.°* As 
identified in TPIE-Qualitative, Triple P researchers also have ongoing roles around making the 
Triple P evidence-base accessible and usable to local implementation stakeholders and for using 
naturally occurring inplementation efforts as opportunities to test effective implementation 
strategies related to Triple P. 
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Providing Implementation Support to NC Counties Scaling Triple P 


While all co-creation partners have essential roles in the implementation and scale-up of Triple 
P, providing active implementation support directly to system stakeholders is a core function 
particularly related to Triple P America and intermediary organizations. These direct 
implementation support providers are a primary mechanism for contributing to the development 
of local implementation capacity and performance. 


Primary Goals of Direct Implementation Support 


Go Greation Local Implementation 


Triple P System Population-level 
Partner Support Capacity eo Performance Optimization Outcomes 


Figure 3. Direct implementation support role and goals. 


The logic model for Triple P implementation (see a short-handed version in Figure 3) reminds us 
that capacity without performance denies the possibility of implementation optimization and the 
realization of child and family outcomes. Likewise, performance without adequate capacity may 
result in temporary, inefficient, and suboptimal outcomes. One helpful way to think about this is 
that both the pipeline (implementation capacity) and the water flowing through the pipeline as 
intended (implementation performance) are essential. As such, although there are seven 
objectives of implementation support articulated in the first section of this plan, the primary 
goals of direct implementation support providers — such as Triple P America and intermediary 
organizations -— are meaningful contributions to the development of strong local 
implementation capacity and performance. Strong local capacity and performance offer the 
foundation on which the realization of other implementation support objectives and local Triple 
P system optimization goals can be most effectively and sustainably achieved. 


Triple P America and intermediary organizations join the multi-level system of support that is 
already developing across North Carolina for the scale-up of Triple P (see Figure 4). The State 
Triple P Learning Collaborative and local leadership and implementation teams within community 
Triple P coalitions work as internal change agents in this multi-level system of support; they work 
from within state and county service system environments to develop Triple P implementation 
capacity and performance. In contrast, Triple P America and intermediary organizations act as 
external change agents; they work from outside state and county service system environments 
to support the development of implementation capacity and performance.*®>89 
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Carefully designed, proactive, and ongoing implementation support from external change agents 
has been identified as a key component for achieving system-level impact, and is considered most 
effective when it contextualizes implementation strategies for local systems and works at 
multiple levels.1”7577° The following sections offer details about the alignment of support from 
Triple P America and intermediary organizations and a flexible, stage-based approach to the use 
of core practice components for external implementation support to strengthen Triple P 
implementation capacity and performance in NC communities. 
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Figure 4. TPA and Intermediary Organization integration into North Carolina’s multi-level system 
of Triple P implementation support. 


Alignment of Support among Triple P America & Intermediary Organizations 

Figure 5 presents an integrated model of implementation support, aligning more generic local 
implementation processes (i.e., not program specific) with the phases of the Triple P 
Implementation Framework (TPIF).°? Intermediary organizations and Triple P America may work 
in concert to support the blend of generic implementation processes and Triple P-specific 
implementation processes. For example, during the exploration stage,!*?’ lead agencies in NC 
communities and their partners will benefit from assessing community wellbeing needs and 
community system members’ readiness to implement practice or program changes. These 
generic implementation processes may be supported by intermediary organizations as needed. 
Once needs and readiness for implementation are clarified, contact may begin between the lead 
community agency and Triple P America (TPA) to assess the fit of Triple P with identified 
community needs and local readiness (i.e., the engagement phase of TPIF). If a decision is made 
to move forward, the organizations develop written agreements for training and other support 
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from TPA (i.e., the commitment and contracting phase) and then progress to the implementation 
planning phase, during which an intermediary organization may also be involved for co-creation. 
Additional examples of an integrated approach to supporting generic and Triple P-specific 
implementation activities are provided in Table 1. 
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Figure 5. Aligning Triple P specific and more generic implementation support. 


With attention to both Triple P-specific and generic implementation capacities, external 
providers of implementation support take a building-block approach across stages of 
implementation to strengthen individual and organizational abilities for the effective use of Triple 
P. Of course, though stages are helpful for conceptualizing the implementation process, 
implementation and scale-up are widely recognized as dynamic, nonlinear processes involving 
multiple decisions, not a single event that occurs over time. Triple P America and intermediary 
organizations need to be proficient at handling the complex entanglement of natural 
implementation processes. 
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IMPLEMENTATION STAGES*””” and EXAMPLE ACTIVITIES 


Exploration 


Installation 


Initial Implementation 


Full Implementation 


Generic 
Implementation 
Activities 


Assessing community 
wellbeing needs 
Assessing system 
readiness to implement 
change 

Assessing current system 
implementation capacity, 
and planning to 
strengthen gaps and 
manage challenges 
Setting up leadership and 
implementation teams 


e Professional development 


to use and support active 
implementation strategies 
Community coalition 
capacity development of 
implementation 
infrastructure (e.g., 
practitioner selection, 
training, coaching 
systems; local quality and 
outcome monitoring 
systems; linking 
communication protocols) 


Identifying and addressing 
adaptive implementation 
challenges 

Strengthening coalition 
and multi-level systems 
environments 

Using process and 
outcome data to improve 
overall implementation 
capacity and performance 


Institutionalizing overall 
implementation capacity 
and performance 

Local coalition-regulation 
of ongoing 
implementation and 
program optimization 
Consideration of how to 
align or add additional 
evidence-based programs 
and practices to meet 
community goals 


Triple P-Specific 
Implementation 
Activities 


Clarifying potential fit for 
Triple P (e.g., target 
population, workforce) 
Clarifying capacities 
needed for chosen Triple 
P levels, formats, and 
goals 


Receiving high-quality 
Triple P training 

Meeting Triple P 
accreditation standards 
Establishing Triple P peer 
support networks (PASS 
Model that builds 
collective regulation) 


Facilitating access and 
engagement for Triple P 
family services 
Delivering Triple P 
programs to families 
Evaluating Triple P 
delivery and refining 
practices 

Using data to improve 
organizational support for 
implementing Triple P 


Building linkages across 
Triple P levels and 
organizations 

Sustaining service 
delivery and support 
processes 

Examining and enhancing 
population-wide impact 


Table 1. Examples of generic and Triple P-specific implementation activities and supports by stages. 
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A Stage-Based Approach to Supporting the Scale-up of Triple P in NC Communities 

Aldridge, Brown, and Bumbarger have proposed a core set of practice components for external 
implementation support. Similar to the need to flexibly draw on identified implementation 
strategies,°°°? core practice components of implementation support might be differentially used 
across stages of implementation. Aldridge et al.’s arrangement of practice components by stage 
of implementation support is presented in Figure 6.2” When core practice components are flexibly 
yet intentionally used over time, external implementation support offers gradual and ongoing 
contributions to strengthen local implementation capacity and performance while ensuring local 
system ownership of the process. 


Although Triple P America’s implementation consultants utilize many of these core practice 
components, in the sections that follow we discuss the necessary incorporation of these practice 
components for external implementation support within the stage-based activities of 
intermediary organizations and NC community Triple P coalitions as they work together to scale 
the Triple P system. Recommended implementation support tools and measures of formative 
and summative implementation outcomes are presented in appendices I-III, and will be included 
in separate resource materials and made available on the NCIC-TP website 


(http://ncic.fpg.unc.edu) as finalized. 


Readiness & Exploration (Exploration Stage) 


Building collaborative |=) 9=12i3) Development (Installation State) 
relationships 


Implementation 
professional 
development for 
individuals 


Supported Performance (Initial Implementation) 
Assessment of aS 
community 
prevention goals and 
current 
implementation 
processes 


Exposure of the 
system to the full 
work of implementation | Reinforcing coalition- 

regulation of 
implementation 
processes 


Coalition 
implementation 
capacity 
development 


Supportive behavioral 
coaching for individuals 
and teams 


Facilitation of local 
implementation 
planning 


Transition of the 
implementation support 
role 


Facilitation of collective 
learning and rapid 
problem solving 


Figure 6. Implementation support core components by stages of implementation support.t 


Readiness & Exploration 

Creating ongoing opportunities for readiness and exploration work within communities intending 
to or already scaling-up Triple P was a key recommendation from county and statewide Triple P 
stakeholders who participated in TPIE-Qualitative.°! High quality readiness and exploration 
processes have been associated with greater implementation success and efficiency in later 
stages of implementation, including for Triple P.% 
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If not already completed, some intermediary organizations may be able to support community 
leaders with local needs assessments, which will ground discussions about the adoption of Triple 
P in local health and wellbeing data and set-up key indicators of population-level success that 
may be monitored over time. Additionally, Triple P fit and feasibility with local community needs 
and coalition partners should be considered, collaboration between co-creation partners must 
be reinforced, local leaders’ sense of change commitment and change efficacy with Triple P needs 
to be gauged, and resources and supports required to sustainably scale the Triple P system should 
be determined. Tools and measures that can support these activities are presented in Appendices 
I-III. 


Once readiness has been established and decisions made to move forward with Triple P and 
external implementation support partners, three core practice components for external 
implementation support are essential for intermediary organizations as they support the creation 
of a local foundation for Triple P success: (1) building collaborative working relationships, (2) 
assessment of community prevention goals and current implementation processes, and (3) 
facilitation of the community coalition’s implementation planning. 


Building collaborative working relationships. The development of collaborative relationships 
between providers of external implementation support and local system stakeholders has been 
one of the most widely discussed factors in relation to high quality implementation support.17°” 
99,62-64,70,89,95-97 In particular, Katz and Wandersman propose seven relationship characteristics 
that are important between providers of external implementation support and support 
recipients: trust, respect, collaboration, adjusting to readiness, strengths-based, autonomy- 
supportive, and rapport.°® 


Intermediary organizations need to proactively foster such relationship characteristics with 
community Triple P coalitions and their co-creation partners. In part, this can be facilitated early 
by collaborative conversations about, and assessments of, community prevention goals and 
strengths and needs of coalition implementation practices. During these conversations, 
intermediary organizations can reinforce and build on existing strengths and be transparent 
about how they can, and cannot, be helpful to community Triple P coalitions. Where coalitions 
may have needs with which the intermediary organization cannot help, brokering connections 
with new co-creation partners or resources can add value to the relationship.?’ Additionally, 
social interactions between collaborative partners, such as periodic social meetings and other 
events that bring people together (often around food), are cited as essential intangibles 
necessary to nurture connections on which social change efforts may rely.?? 


Assessment of community prevention goals and current implementation processes. Collecting data 
about local goals and capacity allows providers of external implementation support to 
accommodate communities’ needs and resources in a way that recognizes current progress and 
enables a strengths-based approach.°® During exploration-stage assessment activities, several 
discussion protocols and assessment instruments related to local implementation capacity and 
performance may be helpful and are presented in Appendices I-lIll. The importance of using 
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specific assessments of need to tailor implementation support and implementation planning, 
rather than relying only on global assessments, has been documented in the context of advancing 
community-wide prevention efforts.°%° It is strongly recommended that intermediary 
organizations and coalition leaders incorporate specific measures of implementation capacity and 
performance and not rely solely on discussion tools or other global inquiry protocols. 


Facilitation of the community coalition’s implementation planning. In preparation for the next 
stages of implementation, Triple P America, intermediary organizations, and community Triple P 
coalition leaders will benefit from the development of a local implementation plan, grounded in 
identified community strengths and needs and guided by strategic, evidence-informed strategies 
for implementation and scale-up. Because of the technical nature of these plans and the likely 
benefits of community Triple P coalition leaders to be supported during their development, 
intermediary organizations and Triple P America, working in partnership, may facilitate the 
generative process. Local implementation plans may include details related to several features 
of the NCIC-TP implementation support logic model (see Figure 1, above), such as: 


e Target goals for, and plans for measuring and monitoring, local population-level 
outcomes; 


e Target goals for, and plans for measuring and monitoring, Triple P system 
implementation outcomes (e.g., fidelity, reach, accessibility, system alignment); 


e Plans for the development of local implementation capacity and the local 
implementation performance needed to meet target goals for Triple P delivery and 
population-level outcomes (i.e., linking leadership and implementation teams within the 
community coalition and their alignment within the state’s multi-level system of 
implementation support; professional development infrastructure; quality and outcome 
monitoring systems; and media and networking capacity); and 


e Plans for involving co-creation partners to support and participate in the development of 
local implementation capacity and overall coalition sustainability. 


The implementation plan should establish a clear, direct connection between the Triple P 
system’s underlying logic model or theory of change and the benchmarks that signify high-quality 
Triple P delivery across acommunity. Furthermore, several supporting implementation, practice, 
and policy resources (e.g., MOUs, data-sharing plans, peer support network plans, fidelity 
monitoring plans, local Triple P Stay Positive media campaign plans) may be acquired and 
adapted from the North Carolina Triple P Learning Collaborative or may otherwise need to be 
developed as a result of the local implementation plan. 


Local implementation plans also allow intermediary organizations to develop their thoughts for 
responsively supporting community Triple P coalitions. In this way, external implementation 
support plans can likewise be developed, informed by identified strengths and needs in 
community implementation capacity and locally established implementation goals. Through 
discussion with community Triple P leaders, intermediary organizations can select a series of 
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professional and coalition capacity development strategies that may best fit local team 
structures, needs, and preferences for external implementation support. 


As a core part of the implementation planning process, the community Triple P implementation 
team should be formally identified. This team is often the ongoing point of connection for 
intermediary organizations and Triple P America Implementation Consultants to support Triple P 
implementation and scale-up in the community. Community Triple P implementation teams, 
often led by one or two Triple P coordinators, are responsible for day-to-day support of the 
community Triple P coalition and its member service agencies. Among other activities, at least a 
portion of the team may assure day-to-day functions for active implementation and scale-up at 
the coalition-level, including:® 


e Assessing and creating ongoing buy-in and readiness within and across Triple P service 
agencies and community prevention systems; 


e Installing, ensuring the aligned operation of, and _ sustaining cross-coalition 
implementation infrastructure and best practices to support Triple P delivery; 


e Developing and implementing coalition-level action plans to manage stage-based work; 


e Ensuring the use of data, including Triple P fidelity and outcome data, within service 
agencies and across the community Triple P coalition for continuous quality and outcome 
improvement; 


e Involving key partners and community members, including the children and parents 
being served, in Triple P implementation support activities and decision-making for 
community system improvement; 


e Organizing and directing the day-to-day flow of information across the coalition and, as 
needed, to the North Carolina Triple P Learning Collaborative to support local Triple P 
implementation and scaling; and 


e Identifying and addressing coalition implementation barriers and ensuring the spread of 


solutions across the coalition to support successful Triple P implementation and scaling. 


Although implementation team structures may vary according to local context and resources, at 
their core, implementation teams may benefit from: 


e being real, organizationally recognized teams;?° 


e having at least three members (though observations from TPIE suggest that community 
coalition Triple P implementation teams may benefit from as many as four to seven 
members with 3.0+ FTE dedicated across the team including at least one or two full time 
coordinators);°1227°0,98 and 


e having the following experiences and embedded team competencies:* 2 
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© experience creating and managing systems changes, informed by data, to 
support the implementation and scale-up of an innovation; 


o fluency with Triple P and quality benchmarks for Triple P implementation and 
scale-up across a community coalition; and 


o fluency with the use of evidence-informed, active implementation strategies. 


Embedding a local evaluator or data manager within the coalition implementation support team 
may also be of benefit, and is being done in several Triple P coalitions across the state. 


More broadly, the community Triple P implementation team may work in partnership with 
community Triple P coalition leadership, which should also be clearly identified as a core part of 
the implementation planning process, to support collective impact backbone functions on behalf 
of the community Triple P coalition, including:73 


e Providing overall strategic direction, 

e Facilitating dialogue between partners, 
e Managing data collection and analysis, 
e Handling communications, 

e Coordinating community outreach, and 


e Mobilizing funding. 


Observations from TPIE evaluation results and feedback from cross-system partners during TPIE- 
Qualitative greatly suggested the importance of strong community Triple P coalition leadership 
and implementation teams for local success.°?>+ 


In conclusion, it should be recognized that although all co-creation partners have a role in 
supporting the time, space, and resources needed for a robust exploration process, intermediary 
organizations may be uniquely positioned to reinforce these activities and, in doing so, create 
more effective and efficient engagement processes for other co-creation partners involved. 


Capacity Development 

Following the exploration process, intermediary organizations partner with community Triple P 
coalition leaders and implementation teams to strengthen local implementation capacity 
through strategies mutually established during the exploration process. Two core practice 
components for external implementation support may be essential for intermediary 
organizations during this stage: (1) professional development for individuals to utilize effective 
implementation strategies, and (2) community Triple P coalition implementation capacity 
development. 


Professional development for individuals to utilize effective implementation strategies. 
Intermediary organizations contribute to the professional development of local Triple P leaders 
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and implementation team members so they can confidently support the implementation and 
scale-up of Triple P using active implementation strategies. Recent evidence suggests that when 
community coalition members better understand models of evidence-based program support 
within community coalitions, they may better support evidence-based program delivery with 
fidelity.2° Professional development needs may vary between community Triple P coalition 
leaders and implementation teams. For example: 


e Community Triple P coalition leaders may need to reinforce their adaptive leadership 
skills, strengthen resources for engaging community members in local implementation 
activities, develop understanding of common barriers and facilitators to the successful 
scale-up of Triple P, and have a clear understanding of how to ensure that community 
Triple P coalition policies and practices are in alignment with evidence-informed 
implementation practices. 


e Community Triple P implementation team members may need to develop skills related 
to increasing coalition partners’ readiness and buy-in, know the intricacies of installing 
implementation infrastructure across the community coalition, have fluency in Triple P 
interventions and active implementation strategies, and be skilled in managing action 
plans and local evaluation and improvement systems. 


In addition, needs may vary within groups of local Triple P leaders or implementation team 
members, necessitating adaptive professional development strategies that support a range of 
prior experience and knowledge. 


To support professional development in active implementation for individuals, intermediary 
organizations may draw from the broad array of implementation science frameworks available 
in the research or professional literature. However, sticking to one or two frameworks for 
consistency of messaging and terminology may facilitate learner development. NCIC-TP largely 
refers to the Active Implementation Frameworks!*77 and their related literature, as well as to the 
literature about community-wide scale-up of evidence-based prevention programs (e.g., 
Communities that Care, PROSPER, Getting to Outcomes, and Collective Impact). In addition to 
the relevance of these literatures to the Triple P Implementation Framework,®? these literature 
bases greatly informed the development of the NCIC-TP Implementation Support logic model and 
will inform NCIC-TP tool and resource development. 


Regardless of the literature or frameworks chosen, intermediaries have a responsibility to ensure 
their local usability. This means that implementation science knowledge and skills should be 
teachable, learnable, doable, repeatable, and assessable in practice®’ as well as locally responsive 
and relevant. 


Community Triple P coalition capacity development. Across the four areas of local implementation 
capacity described in the NCIC-TP Implementation Support Logic Model (refer back to Figure 1), 
intermediary organizations may contribute to the development of organizational and team 
structures, resources and abilities, and policies and practices to support implementation. 


Leadership & Implementation Teams. As discussed throughout this plan, leadership and 
implementation team structures, linked within and across levels of community service systems, 
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are considered and have been demonstrated to be key features of evidence-based program 
implementation and scale-up.2*7° To link leaders and teams across community service systems, 
the formation of community coalition structures, which may utilize lead or backbone 
organizations to support well-defined local service agency collaboratives that share resources 
and address common goals, has been utilized as a key strategy.24192099,100 Recently, the 
emergence of the collective impact literature has offered key principles for cross-sector 
community collaborations.1301-103 Coalition-based approaches to the scale-up of Triple P may 
also provide a good way to ensure opportunities for cross-agency interaction and support, which 
was suggested as an important factor for successfully supporting the scale-up of Triple P by cross- 
system respondents in TPIE-Qualitative.°* 


Intermediary organizations, in consultation with Triple P America as needed, may work with 
community Triple P coalition leaders and implementation teams to contribute to the design and 
documentation of coalition principles and practices, support the development of backbone 
organization capacity’? within the lead agency, and develop and utilize criteria for the 
selection of local service agencies to participate in the Triple P coalition. 


Beyond the organization and expansion of coalition structures to support communitywide Triple 
P scale-up, formalizing Triple P coalition leadership and implementation teams identified during 
the readiness and exploration stage, organizing Triple P service agency leadership and 
implementation teams as agencies join the coalition, and linking these teams together across 
the coalition structure may be important during this stage of support.’?141”?7 TPIE results 
indicated that more fully developed and linked leadership and implementation teams within 
Triple P service agencies were significantly associated with agency continuation of Triple P 
implementation during the two-year evaluation period.°*° Tools and measures relevant to these 
activities are detailed in Appendices I-III. 


Practitioner professional development infrastructure. Professional development infrastructure to 
support coalition Triple P practitioners to sustainably deliver Triple P interventions with fidelity 
and appropriate flexibility includes the following:>®27 


e Practitioner recruitment and selection infrastructure. Intermediary organizations may 
work alongside Triple P America and coalition leadership and implementation teams to 
establish policies and practices for recruiting and/or selecting community service 
practitioners to deliver Triple P programs. Practitioner selection criteria may vary across 
Triple P programs. Regardless, formalizing clear coalition policies and practices that 
integrate implementation best practices for this core implementation component may 
support more successful and sustainable selection outcomes. 


Results from TPIE indicated that selecting only one Triple P practitioner within an agency 
dramatically increased the risk that the agency would not continue to support Triple P 
over time.°° Agencies that continued implementation across TPIE’s evaluation period 
had, on average, over three Triple P practitioners. This sort of clustering of practitioners 
is also reflected in other implementation science literature.?? These findings did not 
pertain to independent or solo practitioners (often therapists in private practice). 
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Regardless, coalition policies and practices might reflect clustering in an effort to ensure 
that a sufficient number of Triple P practitioners are selected and maintained within 
service agencies to support sustainment of Triple P implementation. 


Practitioner Triple P training infrastructure. Triple P America plays a strong role in 
training community service practitioners to deliver Triple P. However, intermediary 
organizations may work alongside Triple P America and Triple P coalition leadership and 
implementation teams to establish coalition policies and practices that align with, and 
reinforce, Triple P America’s training practices. 


Triple P practitioner coaching infrastructure. Triple P’s model of ongoing practitioner 
coaching following Triple P accreditation requires laying infrastructure for coalition 
and/or agency peer support networks. Triple P’s Peer Assisted Supervision and Support 
(PASS) model offers principles and practices to ensure that peer support networks are 
sufficient to expect intended coaching outcomes for Triple P practitioners. 


Intermediary organizations may work alongside Triple P America and coalition leadership 
and implementation teams to lay the infrastructure for Triple P peer support networks. 
Alternate coaching infrastructure that integrates implementation best practices might 
need to be considered in addition to or instead of peer support networks if feasibility or 
appropriateness concerns arise with the PASS model. 


Results from TPIE suggested that infrastructure to support Triple P practitioners’ ongoing 
coaching following accreditation was the area most in need of development across 
participating counties.°° Practitioners’ ongoing receipt of coaching following 
accreditation may be particularly important to sustain the reach of Triple P interventions 
within a community (by increasing the likelihood that practitioners will actually deliver 
Triple P) and to support the delivery of Triple P interventions with fidelity.2%22932,104 


Quality and outcome monitoring systems. Quality and outcome monitoring systems to support 
implementation and practice improvement across community Triple P coalitions include the 
following:>®7 


Fidelity assessment infrastructure. The delivery of programs with fidelity has consistently 
been linked to increased likelihood of program outcomes.?7”78105106 However, the 
measurement and achievement of program fidelity in applied settings has often been 
challenging.?°41107,108 Evidence from TPIE does not suggest otherwise: results indicated 
that infrastructure to support Triple P fidelity assessment was in particular need of 
development across participating counties.°° 


Intermediary organizations may work alongside Triple P America, Triple P researchers and 
program developers, and Triple P coalition leadership and implementation teams to 
ensure the availability of practical fidelity assessment instruments for use in community 
service settings and to establish coalition policies and practices that align with, and 
reinforce, Triple P America’s quality assurance practices. 
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e Decision support data systems. Using data to continuously improve implementation and 
program delivery may lead to higher quality services, greater likelihood of intended 
outcomes, and program sustainability.2”7°1° Diagnostic and evaluative capacity is a 
necessary component of engaging in data-based improvement activities.*? Although 
results from TPIE indicated that decision support data system infrastructure was in strong 
shape at coalition-levels in participating counties (likely due to requirements for 
participation in the state Triple P evaluation), results simultaneously suggested that 
additional development of decision-support data system infrastructure was needed 
across Triple P service agencies.° 


Intermediary organizations may work alongside Triple P America and Triple P coalitions 
to identify key data constructs that will be important for decision-making and 
performance improvement; develop practical data collection, analysis, and reporting 
protocols; develop policies and practices that reinforce the use of data among leadership 
and implementation teams for decision-making and improvement; and ensure that data 
coalition and evaluation processes align with state Triple P evaluation and funder 
requirements. Consideration of both implementation outcomes and population-level 
outcomes, as described in the final elements of the NCIC-TP Implementation Support 
Logic Model (refer to Figure 1, above), may be important. 


e Leadership and implementation teams & practice-policy communication cycles. 
Although the development of leadership and implementation teams has already been 
discussed, it is important to note their relevance to two other core implementation 
components for quality and outcome improvement: facilitative administration and 
systems intervention.??’ Facilitative administration practices relate to the use of 
information about agency/coalition policy and practice facilitators and barriers to 
improve the implementation of Triple P. Systems intervention practices relate to the use 
of information about Triple P successes and larger systems needs to improve and sustain 
the implementation of Triple P. In this way, they are flip sides of the same coin: changing 
internal policies and practices (facilitative administration) vs. influencing external 
environmental contexts and external systems policies and practices (systems 
intervention). Linking leadership and implementation teams together with front-line 
practitioners and with external policymakers to create practice-policy communication 
cycles is an important part of developing capacity for facilitative administration and 
systems intervention practices. 


Although results from TPIE indicated that facilitative administration and systems 
intervention infrastructure was in good-to-strong shape at coalition-levels in participating 
counties (possibly aided by coalition-based approaches), results simultaneously 
suggested that additional development of facilitative administration and systems 
intervention infrastructure was needed across Triple P service agencies.°° Intermediary 
organizations may work alongside Triple P America and coalition leadership and 
implementation teams to ensure that infrastructure and best practices for facilitative 
administration and systems intervention are embedded at all levels. 
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Media and networking capacity. As previously mentioned, developing media and networking 
strategies to mobilize knowledge and behavior change appears to be an important factor in 
achieving population-level outcomes.***3-*? The Triple P system is unique in that it offers Stay 
Positive media strategies that can be adopted, combined into a local media campaign, and 
strategically deployed within community social networks to: 


e Increase the visibility, accessibility, and reach of Triple P in the local community; 
e Offer normative information about child development; 
e Destigmatize the need for parenting support; and 


e Introduce social learning and modeling opportunities into the community at scale. 


Intermediary organizations may work alongside Triple P America, statewide Stay Positive media 
supports, and Triple P coalition leadership and implementation teams to strategically develop 
and implement a local Stay Positive media campaign based on community preferences, 
demography, geography, social networks, and other characteristics. 


Social networking analysis techniques can be helpful in mapping the social networks of 
community Triple P coalition members to inform strategic placement of Stay Positive media 
strategies and accelerate word-of-mouth dissemination of Triple P information. Valente and 
colleagues*? describe and provide some simple measures of social networks that can be used to 
monitor and improve social networks in the local community. 


In conclusion and across all areas of community Triple P coalition capacity development, it is 
important to note that partnerships and regular communication with the North Carolina Triple P 
Learning Collaborative and statewide funders will help ensure the alignment of local 
implementation capacity and measurement efforts with statewide Triple P system activities. 


Supported Performance 

As community Triple P coalitions’ implementation capacities are strengthened and they begin to 
apply their resources and abilities to deliver Triple P programs across the community, 
intermediary organizations may serve coalitions well by closely supporting initial performance 
efforts. In fact, the initial implementation stage is often referred to as the “awkward stage” 
because new system behaviors often come into conflict with longstanding system habits and 
adaptive challenges become fully apparent in the midst of the push to perform.!?° Three core 
practice components for external implementation support may be essential for intermediary 
organizations during this stage: (1) exposure of the coalition to the full work of implementation, 
(2) supportive behavioral coaching for individuals and teams, and (3) facilitation of collective 
learning and rapid problem solving. More than any other practice components of implementation 
support, these three must work in close concert. 


Exposure of the coalition to the full work of implementation. During the stage of supported 
performance, it becomes important for community Triple P coalition leaders and implementation 
teams to fully test out their new resources and abilities to support Triple P implementation and 
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scale-up. The tendency for coalition leaders and members to fall back into familiar, longstanding, 
and often insufficient implementation habits and fragmented systems practices can be great, 
particularly as comfort with new practices and partners remains emergent. But leaning on newly 
organized coalition resources and abilities as the adaptive work of implementation and scale-up 
escalates can provide essential opportunities for individual and collective learning. These learning 
opportunities and collective risk-taking are essential to identifying areas of required 
improvement and sufficiently sustaining system change.°” Intermediary organizations need to 
help motivate staff from all levels of the coalition — and co-creation partners — to fully lean into 
the discomfort of new ways of work that were established during the capacity development stage 
of implementation support. 


Supportive behavioral coaching for individuals and teams. As community Triple P coalition leaders 
and implementation teams fully engage in the practice of Triple P implementation and scale-up, 
intermediary organizations can support competent implementation practices and nurture 
confidence by providing supportive behavioral coaching.®*”°%° Intermediary organizations take 
on the responsibility for ensuring that implementation and scale-up work moves forward without 
the coalition or its members becoming overwhelmed or losing their collective sense of efficacy.1?° 


Chilenski and colleagues’? describe the use of “an empowerment approach that includes asking 
open-ended questions which encourage [team leaders and members] to critically reflect on their 
knowledge and experience, encouraging teams to brainstorm pros and cons when assessing 
solutions to the many challenges that they will face, providing positive yet constructive feedback, 
helping the team leader and team set realistic goals, and encouraging team-centered 
accountability” (p. 26). Eiraldi and colleagues®> note the importance of directly observing local 
implementation processes to provide personalized feedback and problem solving. Similarly, Ray 
and colleagues found that on-site coaching was particularly important, as external providers of 
implementation support had difficulty achieving skill change and improvement via phone or 
email communications. This core practice component for external implementation support is a 
parallel process to coaching practitioners’ delivery of new front-line programs and practices, 
which has been found to increase the use and quality of innovation delivery.277°224 


Facilitation of collective learning and rapid problem solving. Finally, intermediary organizations can 
support community Triple P coalition leaders and implementation teams in this stage of 
implementation support by facilitating collective learning and rapid problem solving.17?7°3,9°114 
Triple P America Implementation Consultants also may play an important, program-specific role 
in these activities, since barriers and challenges may arise related to either the Triple P system or 
to the implementation strategies being used to support the Triple P system. Intermediary 
organizations may facilitate problem solving by connecting Triple P coalition leaders and 
implementation teams with Triple P America Implementation Consultants or other outside 
knowledge or supports.°’ Alternatively, internal problem solving can be facilitated by the use of 
Plan-Do-Study-Act (PDSA) and other continuous’ quality improvement — cycle 
techniques.?”°?®)73119 Intermediary organizations may help Triple P coalition implementation 
teams design and implement PDSA cycles and may provide coaching on the PDSA process, though 
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accountability for learning and problem solving needs to reside within the local coalition itself to 
support local ownership of progress. 


Local Coalition-regulation 

As community Triple P coalitions’ implementation performance stabilizes and intended 
implementation and program outcomes begin to appear, intermediary organizations may begin 
to consider tapering their support. Two core practice components may be essential for 
intermediary organizations during this stage: (1) Reinforcing coalition-regulation of 
implementation processes and (2) transition of the implementation support role. 


Reinforcing coalition-regulation of implementation processes. During this stage of implementation 
support, coalition leaders and implementation teams should be expressing a decreased need for 
intensive support from intermediary organizations and be ready to locally manage the continual 
improvement of their implementation structures and processes. Sanders and Mazzucchelli* 
detail five key elements of parent self-regulation hypothesized to be built through Triple P 
interventions: self-management tools, self-efficacy, personal agency, self-sufficiency, and 
problem-solving. Here we reframe these principles as coalition-regulation principles that can be 
nurtured by intermediary organizations throughout all stages of their engagement with 
community Triple P coalitions, but should be particularly reinforced during this last stage: 


e Collective-management tools: During earlier stages of implementation support, 
intermediary organizations may have contributed to the development of coalition team 
structures, protocols (e.g., professional development plans, data plans, communication 
protocols), measures, and other tools to support local management of Triple P 
implementation and scale-up. In this final stage of support, intermediary organizations 
need to reinforce the ongoing integration and use of these collective-management tools 
by coalition leaders and implementation teams. Intermediary organizations should also 
encourage coalition leaders and implementation teams to regularly update related 
documents, such as team terms of reference, in the case of system or staff changes. 


e Collective-efficacy: Mentioned earlier in relation to the practice principle local ownership 
of progress, collective-efficacy represents coalition leaders’ and implementation team 
members’ perceptions of their collective abilities to use new implementation structures 
and practices to attain desired implementation outcomes. Reinforcing coalition leaders’ 
and implementation team members’ sense of collective-efficacy during this final stage of 
implementation support can enable them to confidently work together to continue to 
make progress and improvement in community prevention and wellbeing efforts. 


e Collective agency: Intermediary organizations may also reinforce coalition leaders’ and 
implementation teams’ sense of collective agency in determining local Triple P goals and 
improving local implementation capacity and performance to reach those goals. This 
ensures that coalition leaders and implementation teams take responsibility for, feel 
ownership of, and have influence over the actions that support coalition changes and 
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implementation practices. It also facilitates intermediary organizations’ transition away 
from an intensive support role. 


e Collective-sufficiency: Reinforcing coalition leaders’ and implementation teams’ 
collective-sufficiency during this stage does not mean increasing their isolation from 
support. Instead, coaching during this time can focus on reinforcing coalition leaders’ and 
implementation teams’ membership within larger support networks, and enable them to 
independently solve problems with ongoing support from intermediary organizations and 
other co-creation partners as needed. 


e Adaptive problem solving: By building adaptive leadership skills with coalition leaders 
and implementation team members, intermediary organizations inherently reinforce 
adaptive problem solving, which by definition involves giving work back to people 
collectively instead of driving decision-making from above.’? Ongoing adaptive leadership 
and problem-solving capabilities may be necessary ingredients for the sustainability of 
EBPs due to the fluidity and complexity of community systems environments. 


Transition of the implementation support role. If intermediary organizations take too great of a 
responsibility for ensuring local implementation processes, fading their support role can be 
difficult and also transferring more implementation leadership to coalition leaders and 
implementation teams has a greater likelihood of failing.’”°111"4 In some situations, it may be 
helpful for intermediary organizations to develop an explicit transition strategy with community 
Triple P coalitions. In other situations, the realization that coalition leaders and implementation 
teams have started coalition-regulation processes signals the transition of the intermediary 
organization out of the regular flow of local implementation work. 


There will likely be future circumstances that create vulnerability for sustaining effective 
implementation and scale-up within community Triple P coalitions — times of turbulence and 
change in community or statewide environments, changes in leadership, and changes in the fit 
or feasibility of the Triple P system. Intermediary organizations can be proactive by discussing 
these potentials with coalition leadership and implementation teams prior to transitioning away 
from local implementation processes and explore how these circumstances might be addressed 
should they arise. There are occasions when re-engaging intermediary organizations may be 
particularly appropriate and offer a constructive approach. 


Finally, the collaborative partnership and mutual learning that has taken place between 
intermediary organizations, community Triple P coalition members, and other co-creation 
partners should be recognized and celebrated. Because intensive relationships between external 
providers of implementation support and local partners are likely to last across several years, 
ensuring space and time for healthy reflection and celebration can strengthen the partnership 
even as it takes a new, less intensive form. This also promotes the likelihood that coalition 
members will continue to reach out to intermediary organizations for ongoing needs or share the 
benefits of engaging with the intermediary organization with statewide colleagues. 
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A Core Story of Implementation Support, Flexibly Applied 

In summary, NCIC-TP suggests that the development of implementation capacity, tailored within 
local contexts, may be most effectively served by the common-elements approach detailed 
through this implementation support plan rather than by a prescriptive approach to external 
implementation support. Because the process of implementation occurs in complex and dynamic 
environments and depends on local factors such as resources, stability, and timing, 
implementation support activities may need to be adapted throughout the support period. 


NCIC-TP therefore offers a core story of implementation support that can be flexibly applied 
across NC community Triple P coalitions. As depicted in Figure 6, intermediary organizations 
adaptively provide implementation support by responsively integrating practice principles and 
core practice components to tailor the support process, which has been widely discussed as a 


key factor for successfully contributing to local implementation capacity and performance.?”>%*" 
65,110,111,114,115 
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Figure 6. Unpacking the external implementation support process for contributing to the 
development of local implementation capacity and performance. 
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APPENDIX |: Recommended Tools to Support Implementation & Scale-Up Processes 


During the implementation and scale-up process, several tools may be useful to community Triple P coalition leaders and 
implementation teams as they develop local implementation policies and practices. In addition, external providers of implementation 
support may find these tools helpful as they work closely with coalition leaders and teams to support local decision-making and 
documentation. 


Tool Name Brief Description Recommenged Location 
Stage or Frequency 

The Hexagon The National Implementation Research Coalition Readiness & http://implementatio 

Tool Network’s Hexagon Tool is a review and_ leaders Exploration; as n.fpg.unc.edu/resourc 
discussion tool organized around _ six- needed when es/hexagon-tool- 
components (need, fit, resources, evidence, considering exploring-context 
readiness, capacity) that can be used to support additional Triple P 
program selection. This tool can be used both programs 


for initial discussions about the adoption of the 
Triple P system and for ongoing discussions 
about the adoption of additional Triple P 
programs and media strategies within the 


system. 

_NCIC-TP A series of readiness worksheets, adapted from” Intermediary Readiness & Draft worksheet series 
Readiness Romney and colleagues’ Triple P readiness | organizations; Exploration; is being developed by 
Worksheet worksheets,2* can be used to prepare local Coalition | Capacity NCIC-TP 
Series stakeholders for understanding and managing leaders & Development 


expectations for implementing or scaling Triple implementation 
P. These worksheets can assist local teams; Service 
stakeholders to consider requirements and agency leaders 
| practices associated with effective Triple P (particularly the © 
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implementation and identify gaps that may Service Agency 


need to be addressed before moving forward. 


(1) Community Triple P Readiness for 


(2 


~— 


Scaling Triple P: this worksheet can be 
used to determine the presence of 
community structures and practices that 
may indicate level of readiness to 
support communitywide Triple P scale- 
up. 

Community Triple P Coalition Readiness 
to Participate in Intermediary Support: 
this worksheet allows intermediary 
organizations to collect basic 
information about community Triple P 
coalitions and to assess key factors 
related to readiness for partnering with 
an intermediary organization for Triple P 
implementation support. 


(3) Service Agency Readiness __ for 


Implementing Triple P Interventions: this 
worksheet can be used to determine the 
presence of key service agency practices 
that may indicate level of readiness to 
support implementation of Triple P 
interventions. 


This series of five semi-structured discussion 
protocols can be used to facilitate collaborative organizations; 
inquiry and informal assessment of the Coalition 


version) 


Intermediary 


Readiness & 
Exploration; as 


39 


Draft discussion tool 
series is being 


Discussion Tool 
Series 


Technical 
Assistance 
Analysis 
Discussion Tool 


involvement of co-creation partners and the 
four core areas of implementation capacity: 
linked leadership and implementation teams, 
professional development _ infrastructure, 
quality and outcome monitoring systems, and 
media and networking capacity. 


These protocols may be particularly helpful 
when more structured or specific 
implementation assessments are unwarranted, 
as they were written about global details of 
local prevention goals, coalition strategic plans, 
coalition policies and _ practices, and 
implementation successes and needs. 


Despite the utility of these discussion tools, the 
importance of using specific assessments of 
need to tailor implementation support and 
implementation planning, rather than relying 
only on global assessments, has been 
documented in the context of progressing 
communitywide prevention efforts.°° Therefore, 
it is strongly recommended that coalition 
leaders and intermediary organizations not rely 
only on these semi-structured discussion tools. 


Blase’ offers a quantitative assessment of 
required intensity of technical assistance that 
can be used to score ten relevant factors on a 
continuum from basic to intensive technical 
assistance. This discussion tool can help discern 
the intensity of external implementation 


leaders & 
implementation 
teams 


Coalition 
leaders, 
intermediary 
organizations, 


otherwise helpful developed by NCIC- 
during later stages TP 


Readiness & http://challengingbeh 
Exploration avior.fmhi.usf.edu/do 


/resources/document 
s/roadmap_4.pdf (see 
page 4) 
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Creating Team 
Terms of 
Reference 
Worksheet 


Communication 
Protocol 
Worksheet 


support needed to match the degree of change 
being undertaken by a local community. Blase 
acknowledges the need for more intensive 
implementation support and change facilitation 
when there is a significant discrepancy between 
current and desired practice. 


This worksheet, developed by the National 
Implementation Research Network, can be used 
to facilitate discussion around several possible 
components of team terms of reference, 
including vision, goals and objectives, scope and 
boundaries, roles and responsibilities, 
communication protocols, resources, authority, 
deliverables, and implementation — plans. 
Discussion of some or all of these components 
can prepare a team to create a more formal 
terms of reference document. 


This worksheet, developed by the National 
Implementation Research Network, helps 
teams within an agency or across agencies 
establish new communication patterns with 
clear expectations and roles. Beyond linking 
leadership and implementation teams, these 
communication protocols can also be 
established between groups of front-line 


and co-creation 
partners 


Individual 
teams within 
community 
Triple P 
coalitions (e.g., 
coalition 
leadership 
teams, coalition 
implementation 
teams, agency 
leadership 
teams, agency 
implementation 
teams) 


Linked teams or 
groups within 
community 
Triple P 
coalitions 


Capacity 
Development 


Capacity 
Development 


http://implementatio 
n.fpg.unc.edu/resourc 
es/activity-3-4-terms- 
reference-examples- 
and-mock 


See Lesson 9: 
http://implementatio 
n.fpg.unc.edu/module 
s-and-lessons# 


NCIC-TP Locus of 
Responsibility 
Worksheet 


Triple P America 
Training 
Outcome 
Reports 


practitioners and agency or coalition leadership 
to support practice-policy communication 
cycles. 


Clearly establishing responsibility for various 
aspects of implementation can be challenging in 
the context of community coalitions and multi- 
level systems of implementation support.® This 
worksheet allows intermediary organizations 
and community Triple P coalition members to 
discuss and clarify three types of responsibility 
across all implementation core components:>” 
who does it, who assesses it, and who ensures 
it? 


Various aspects of responsibility can be 
assigned to the statewide Triple P supports (i.e., 
the North Carolina Triple P State Leadership 
Team, North Carolina Triple P Learning 
Collaborative, North Carolina’ Triple P 
Evaluation), Triple P America, Community Triple 
P Coalition, Local Triple P Service Agencies, Local 
Triple P Practitioners, or designated as “unclear 
and in need of further review.” 


Triple P America provides training outcome 
reports that detail pre- and post-training learner 
outcomes and participant experience data (e.g., 
satisfaction) for each Triple P training course 
conducted. Also included are participants’ 
accreditation statuses. These reports provide 


Coalition 
leadership 
teams; 
Coalition 
implementation 
teams 


Coalition 
implementation 
teams; Agency 
leaders and 


Capacity Building 


Capacity Building 
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Triple P Peer 
Support 
Checklist 


Stay Positive 
Media 
Strategies 


Collective 
Learning 
Database 


valuable information for coalitions, service 
agencies, and practitioners on the Triple P 
training process. 


This checklist, available from Triple P America, 
allows tracking of key peer support activities 
and monitoring core peer support components 
(e.g., use of audio or video during case 
presentations). 


Stay Positive media strategies include 
Tippapers, informational materials (flyers, 
brochures, and posters), newspaper articles, 
roadside billboards, television and radio spots, 
and Tip Sheets. Individual media strategies can 
be adopted and combined into a local media 
campaign, and strategically deployed within 
community social networks. 


This electronic database allows the 
documentation of identified implementation 
barriers and facilitators, the strategies used to 
address barriers, and other collective learning 
insights as appropriate across the 
implementation and scale-up initiative. 


implementation 
teams 


Coalition 
implementation 
teams; Agency 
implementation 
teams; 
Coalition Triple 
P practitioners 


Coalition 
leadership 
teams; 
Coalition 
implementation 
teams 


Coalition 
leaders, 
implementation 
teams, agency 
representatives, 
and other 
partners 


Capacity Building; 
Supported 
Performance 


Capacity 
Development 


Across all stages 
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APPENDIX II: Recommended Measures of Implementation & Scale-Up 


Several process and outcome measures may be useful as coalition implementation teams monitor implementation and scale-up. 
External providers of implementation support may work closely with coalition implementation teams to utilize these measures and 
may be responsible for facilitating these assessments as noted, needed, or helpful. 


Measure Name Brief Description Respondents Bocommended Location 
Stage or Frequency 
Community Chilenski and  colleagues’!4© community Mixed samples During initial Draft is complete and 
Readiness Scale readiness scale offers a 15-item, four-factor of community readiness activities being processed by 
assessment of community readiness to leaders, service NCIC-TP for posting 
implement large-scale community change agency on website 


projects that involve several community representatives, 
partners. Subscales offer information about parents, and 
community attachment, community initiative, youth 
community efficacy, and community 


leadership. 
Community Triple This scale was adapted by NCIC-TP from Coalition and | Across all stages "Draft is complete and 
P Buy-In Scale Perkins and colleagues’?’” five-item measure agency Triple P being processed by 
of community buy-in for local PROSPER coordinators and NCIC-TP for posting 
initiatives. The scale contains five items that implementation on website 


assess the degree to which influential team members 
community leaders are committed to and (excluding lead 
champion the community Triple P initiative. agency directors 
and service 
agency 
directors) 
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Organizational 
Readiness for 

Implementing 
Change (ORIC) 
Measures 


The Wilder 
Collaboration 
Factors Inventory 


Shea and colleagues’ 10-item ORIC measure?® 
has been adapted by NCIC-TP to offer brief 
measures for assessing readiness to 
implement or scale Triple P. The instrument 
provides scores. related to leaders’ 
commitment to the change process (change 
commitment) and their sense of efficacy that 
change can be _ accomplished (change 
efficacy). 


(1) Coalition Version: This version can be 
used to assess community coalition 
readiness to scale Triple P. 

(2) Service Agency Version: This version 
can be used to assess service agency 
readiness to implement Triple P 
interventions. It may be helpful as 
community service agencies are 
considered for membership in the 
community Triple P coalition. 


This 40-item inventory offers a research- 
based measure that can be used to assess 20 
collaborative factors among community 
agencies and partners involved in emergent or 
existing community Triple P coalitions. 


Coalition leaders 
(Coalition 
Version); Service 
agency leaders 
(Service Agency 
Version) 


Co-creation 
partners; 
Coalition 
leaders, 
including from 
both lead and 
service agencies 


Readiness & 
Exploration 
(Coalition Version); 
Capacity 
Development 
(Service Agency 
Version) 


Across all stages 
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Drafts are complete 
and being processed 
by NCIC-TP for 
posting on website 


http://www.wilder.or 
g/Wilder- 
Research/Research- 
Services/Pages/Wilde 


r-Collaboration- 
Factors- 


Inventory.aspx 


Community 
Capacity 
Assessment for 
Scaling the Triple 
P System of 
Interventions 
(CCA-TP) 


The CCA-TP was originally developed by TPIE 
evaluators to provide an assessment of key 
abilities and related resources in communities 
implementing the Triple P system of 
interventions. For the development of the 
CCA-TP, TPIE evaluators relied heavily on 
previous assessment protocols used to 
measure the capacity of counties or school 
districts to effectively support the 
implementation and scaling of evidence- 
based interventions.1!? 12° The CCA-TP is a 
facilitated group self-assessment that, having 
been further refined since TPIE, now includes 
110 items organized within the following 
eleven indices: 


Fidelity Assessment, 
Decision-Support Data System, 
Facilitative Administration, and 
Systems Intervention. 


a. Coalition Leadership Team, 

b. Coalition Implementation Team, 
c. Prevention System Alignment, 
d. Action Planning, 

e. Recruitment & Selection, 

f. Training, 

g. Coaching, 

h. 

i 

i) 

k. 


Additionally, three summary indices can be 
calculated: 


Coalition leaders 
and 
implementation 
team members 


Semi-annually 
across all stages 
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Draft is complete and 
being packaged by 
NCIC-TP for posting 
on website 


Implementation 
Drivers 
Assessment for 
Agencies 
Implementing 
Triple P 
Interventions 
(IDA-TP) 


a. Coalition Implementation Teams Index 
(indices a-d above), 

b. Coalition Implementation Drivers Index 
(indices e-k above), and 

c. Coalition Sustainability Planning Index 
(combining three specific items). 


To ensure reliable assessment, the CCA-TP 
should be administered by an implementation 
support specialist. 


The IDA-TP was originally developed by TPIE 
evaluators to assess the presence of active 
implementation infrastructure and_ best 
practices among Triple P service agencies to 
support the intended delivery of Triple P 
interventions. TPIE evaluators relied heavily 
on previously established implementation 
drivers assessments and technical assistance 
tools for the development of IDA-TP items and 
scales.!¢4-125 The IDA-TP is a facilitated group 
self-assessment that, having been further 
refined since TPIE, now includes 89 items 
organized within the following eight indices: 


Fidelity Assessment, 
Decision-Support Data System, 


a. Agency Implementation Capacity, 
b. Recruitment & Selection, 

c. Training, 

d. Coaching, 

e. 

I 


Service agency 
leaders and 
implementation 
team members 


Semi-annually 
across all stages 
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Draft is complete and 
being packaged by 
NCIC-TP for posting 
on website 


Team Functioning 
Measures 


g. Facilitative Administration, and 
h. Systems Intervention. 


Additionally, two summary indices can be 
calculated: 


a. Agency Implementation Drivers Index 
(indices b-h above), and 

d. Agency Sustainability Planning Index 
(combining three specific items). 


To ensure reliable assessment, the IDA-TP 
should be administered by an implementation 
support specialist. 


Chilenski and colleagues” utilized five brief 
measures to assess key aspects of PROSPER 
teams’ functioning in their investigation of the 
importance of collaboration between external 
providers of implementation support and 
community prevention teams. 


(1) Team Leadership:  Chilenski and 
colleagues’ eight-item measure of the 
degree to which team _ leadership 
encourages input and consensus, along 
with promotes a_ friendly work- 
environment, originally adapted from 
Kegler and colleagues.”° 


(2 


~— 


Team Culture: Chilenski and colleagues’ 
eight-item measure of team 


Individual teams Capacity Building; 


within Supported 
community Performance; Local 
Triple P Coalition- 
coalitions (e.g., Regulation 
coalition 

leadership 


teams, coalition 
implementation 
teams, agency 
leadership 
teams, agency 
implementation 
teams) 
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Drafts are complete 
and being processed 
by NCIC-TP for 
posting on website 


Triple P Service 
Agency 
Implementation 
Climate Scale 


atmosphere, originally adapted from 
Kegler and colleagues.”° 


(3 


— 


Team Goals: Perkins and colleagues’*?” 
two-item measure of the degree to 
which teams have developed clear 
goals and governance procedures. 


(4) Team Focus on Work: Chilenski and 
colleagues’ five-item measure of teams’ 
work orientation, originally adapted 
from Moos & Moos.??° 


(5) Team Tension: Feinberg and 
colleagues’?2” single-item measure of 
team tension. 


A seven-item measure of agency 
implementation climate was adapted by TPIE 
evaluators from Klein, Conn, and Sorra’s 
implementation climate scale.178 Based on 
data from this measure, TPIE results indicated 
that Triple P service agencies with less 
hospitable implementation climates were at 
greater risk for discontinuing Triple P 
implementation during the TPIE evaluation 
period.°° Klein and colleagues demonstrated 
that their original implementation climate 
scale was associated with leadership and 
management support of innovation 
implementation.128 Therefore, lower scores 
on the Triple P version of the scale may 
suggest a timely need for county Triple P 


Triple P service 
agency 
practitioners and 
staff members 


Supported 


Draft is complete and 


Performance; Local being processed by 


Coalition- 
Regulation 


NCIC-TP for posting 
on website 
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Coalition- 
Regulation 


Social Network 
Analysis 


coalition implementation teams to help re- 
establish service agency leadership and 
implementation teams’ local support for 
Triple P. 


Alternate measures of implementation 
climate are available as well,!*?13° and can be 
considered based on local preferences or 
needs. 


Measures of the five proposed components of coalition-regulation are being explored by the NCIC-TP Team: 


e Collective-management tools 

e Collective-efficacy 

e Collective agency 

e § Collective-sufficiency 

e Adaptive problem solving 
Social networking analysis techniques can be Coalition 
helpful in mapping the social networks of leadership 
community Triple P coalition members to teams; Coalition 
inform strategic placement of Stay Positive implementation 


media strategies and accelerate word-of- teams 
mouth diffusion of Triple P information. 


Capacity 
Development 


Valente and 
colleagues*? describe 
and provide some 
simple measures of 
social networks that 
can be used to 
monitor and improve 
social networks in 
the local community 
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Stay Positive 
Media Tracking 


Triple P System 
Implementation 
Outcomes: 
Accessibility* 


Triple P System 
Implementation 
Outcomes: 


Triple P America has suggested several ways 
to track the performance of Stay Positive 
media strategies, including: 


(1) Semi-annual analytics provided by 
Triple P International that detail 
utilization of the local Stay Positive 
website by practitioners and parents, 
and 


(2 


— 


The Stay Positive Campaign Tracking 
Form, a Microsoft Excel database that 
can be used by coalition 
implementation teams to record details 


about the number, placements, 
estimated reach, and _ intended 
purposes of Stay Positive media 


strategies in the community. 


To monitor the accessibility of Triple P 
interventions with the community, Triple P 
coalitions might survey parents about the 
accessibility of Triple P programs. Reviewing 
the geographic distribution of Triple P 
practitioners across the region may also 
provide helpful information. 


The Prevention System Alignment Index 
within the Coalition Capacity Assessment for 
Communities Scaling the Triple P System of 
Interventions (CCA-TP) provides information 
about the degree to which Triple P 


These are output Supported 


tracking 
measures — 
respondents are 
not applicable 


Community 
parents; 
Community 
service agencies 
and practitioners 


Triple P America 
Performance; Local 

Coalition- 

Regulation 


Supported 
Performance; Local 
Coalition- 
Regulation 


Not applicable 


See CCA-TP row above 
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System 
Alignment* 


Triple P System 
Implementation 
Outcomes: 
Feasibility* 


Triple P System 
Implementation 
Outcomes: 
Appropriateness* 


Triple P System 
Implementation 
Outcomes: 
Acceptability* 


interventions have been adopted in response 
to identified community wellbeing needs, the 
degree to which Triple P service agencies have 
been included to fill key service gaps within 
the community coalition, and the extent to 
which coalition agencies are aligned and 
supported to collaborate. 


The Triple P Feasibility Scale is a five-item 
measure of Triple P implementation 
feasibility, adapted for use with Triple P from 
Weiner, Dorsey, Stanick, Halko, Powell, & 
Lewis.131 


The Triple P Appropriateness Scale is a five- 
item measure of Triple P appropriateness, 
adapted for use with Triple P from Weiner, 
Dorsey, Stanick, Halko, Powell, & Lewis.13+ 
This scale can be used to measure either Triple 
P implementation appropriateness with 
community Triple P coalition practitioners and 
partners or Triple P program delivery 
appropriateness with families. 


Two measures are recommended: 


(1) The Triple P Acceptability Scale is a five- 
item measure of Triple P acceptability, 
adapted for use with Triple P from 
Weiner, Dorsey, Stanick, Halko, Powell, 
& Lewis.131 This scale can be used to 


Triple P coalition 
practitioners; 
Triple P coalition 
partners 


Triple P coalition 
practitioners; 
Triple P coalition 
partners; 
Community 
families 


Triple P coalition 
practitioners; 
Triple P coalition 
partners; 
Community 
families 


Supported 
Performance; Local 
Coalition- 
Regulation 


Supported 
Performance; Local 
Coalition- 
Regulation 


Supported 
Performance; Local 
Coalition- 
Regulation 
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In development by 
NCIC-TP 


In development by 
NCIC-TP 


The Triple P 
Acceptability Scale is 
in development by 
NCIC-TP 


The Caregiver 
Satisfaction 


measure either Triple P implementation 
acceptability with community Triple P 
coalition practitioners and partners or 
Triple P program delivery acceptability 
with families. 


(2) Triple P America’s Caregiver 
Satisfaction Questionnaire provides a 
brief measure of — caregivers’ 
satisfaction with Triple P services they 
have received. 


Triple P System Three approaches to measuring Triple P 
Implementation fidelity include: 

Outcomes: 

Fidelity* (1) Triple P Session Checklists: provided by 


Triple P America, these checklists offer 
a session-by-session way for 
practitioners to track and report quality 
adherence related to Triple P program 
delivery. 


(2 


— 


Caregiver Engagement: coalition Triple 
P practitioners might track and report 
caregiver engagement with Triple P 
program. activities by monitoring 
caregiver participation in, and 
completion of, in-session activities and 
between-session assignments. 


(3 


~— 


Dosage: coalition Triple P practitioners 
might track and report the number of 


Coalition Triple P Supported 


practitioners 


Performance; Local 


Coalition- 
Regulation 
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Questionnaire is 
available from Triple 
P America 


Triple P America 
supplies Session 
Checklists and may 
have strategies for 
measuring caregiver 
engagement and 
dosage 


Triple P System 
Implementation 
Outcomes: 
Reach* 


Triple P System 
Implementation 
Outcomes: Cost* 


Triple P System 
Implementation 
Outcomes: 
Sustainability* 


Triple P sessions that caregivers 
complete as a proportion of the total 
number of Triple P sessions indicated 
for a given Triple P program. 


Triple P coalitions can measure reach as: 


(1) the number of community families who 
receive Triple P interventions compared 
to those who are eligible to receive 
Triple P interventions, and/or 


(2) the number of practitioners (actively) 
delivering the Triple P interventions 
compared to the number trained in or 
expected to deliver Triple P 
interventions. 


Triple P 
practitioner 
contact records; 
Triple P 
practitioner 
training records 


The recent publication from the National Academies of Sciences, 
Engineering, and Medicine, Advancing the Power of Economic 
Evidence to Inform Investments in Children, Youth, and Families, 
describes and provides methods for tracking cost of evidence- 


based programs such as Triple P.”9 


Triple P coalitions can measure sustainment of 
Triple P services by tracking the extent to 
which coalition Triple P service agencies and 
practitioners remain actively implementing 
and delivering Triple P. 


Coalition service 
agencies; 
Coalition Triple P 
practitioners 


Supported Community Triple P 
Performance; Local Coalition records 
Coalition- 

Regulation 


Supported Performance; Local Coalition- 
Regulation 


Supported 
Performance; Local 
Coalition- 
Regulation 


Not applicable 


* Those interested in browsing additional implementation outcome measures may benefit from searching the Society for 


Implementation 
https: 


www.societyforimplementationresearchcollaboration.or: 


Research Collaboration’s 


(SIRC) 


sirc-projects/sirc-instrument 


instrument 


repository at 
-project/. As part of a study funded 
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by the National Institute of Mental Health to advance measurement in implementation science, each measure in the repository is 
rated according to its evidence-base and pragmatism for use in community settings.137133 While a SIRC membership is needed to 
access this repository, interested stakeholders can contact NCIC-TP team members if they do not have access and are unable to 
purchase one. 
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APPENDIX III: Recommended Measures and Records of Implementation Support Quality 


Throughout the support period, external providers of implementation support benefit from collecting and using data about the 
delivery and outcomes of their provision of implementation support to optimize their contributions. Several measures and quality 
assurance tools may be helpful. 


Recommended 


Measure Name Brief Description Respondents Location 
Stage or Frequency 

Brief Alliance Adapting Mallinckrodt & Tekie’s!?4 16-item Coalition leaders Quarterly across all In exploration by 
Inventory for Brief Alliance Inventory (BAI) will be explored and stages NCIC-TP 
Implementation by the NCIC-TP team to provide a measure of implementation 
Support the collaborative nature of the relationship team members 

between external providers of 

implementation support and coalition leaders 

and implementation team members. In its 

present form, the BAI provides information 

about two relationship components: Bonds 

(mutual trust, acceptance, and confidence), 

and Goals/Tasks (mutual endorsement of 

working goals and relevance of associated 

tasks). 
Implementation Chilenski and colleagues’”° seven-item scale Members of Quarterly across all In development by 
Support to describe the degree to which the localteam | external stages NCIC-TP 
Collaboration communicates with and works collaboratively implementation 
Scale and effectively with the implementation support teams 


support team. This scale was originally used 
with PROSPER Prevention Coordinators. 
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Contact with 


External Providers 


of 
Implementation 
Support 


Implementation 
Support Quality 
Assurance 
Checklists 


Individual 
Professional 
Development 
Indicators 


Local 
Implementation 
Capacity & 
Performance 
Outcomes 


Chilenski and colleagues’’° two-item measure 
of the frequency of contact between 
implementation technical assistance 
providers and community teams, originally 
used with PROSPER Prevention Coordinators. 


Forms developed by NCIC-TP to track the 
completion of key activities and document key 
outcomes of the Readiness & Exploration 
stage of implementation support and the 
Transition of Support Role process. 


Pre-training, post-training, and follow-up 
learning indicators should be developed for 
major implementation science training events 
with community leaders and implementation 
team members. Learning indicators should be 
aligned with pre-established learning 
objectives and be appropriate to knowledge 
or skill acquisition. 


Members of 
external 
implementation 
support teams 


Members of 
external 
implementation 
support teams 


Training 
participants, 
usually coalition 
leaders and 
implementation 
team members 


Quarterly across all 
stages 


At the end of 
Readiness & 
Exploration; At the 
end of the external 
implementation 
support period 


Capacity 
Development 


In development by 
NCIC-TP 


In development by 
NCIC-TP 


Will be developed as 
needed by NCIC-TP; a 
database of items 
will be made 
available on the 
NCIC-TP website 
once sufficient 


As the development of local implementation capacity and performance are the primary outcomes of external 
implementation support, the CCA-TP and the IDA-TP can be utilized regularly to monitor the long-term 


effectiveness of implementation support. The CCA-TP, being administered to coalition 


leaders and 


implementation team members, may be the most appropriate and direct measure of the outcomes of external 
implementation support. Though relevant to external implementation support outcomes as well, the IDA-TP may 
be more fitting to measure the outcomes of implementation support from the coalition implementation team 
to Triple P service agencies. See the prior section on measures of implementation for details about the CCA-TP 


and IDA-TP. 
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Supported 
Performance 
Records 


Implementation 
Support 
Performance 
Monitoring 


Supported Performance Record: This form 
provides a quality assurance record of 
specifically designed learning experiences 
collaboratively planned between external 
providers of implementation support and 
coalition leaders and implementation team 
members. Learning experiences should be 
designed to facilitate specific implementation 
skill demonstration and improvement (e.g., of 
adaptive leadership skills, facilitating the 
development of agency teams’ terms of 
reference). The form also allows the 
documentation of lessons learned and 
coaching strategies that were responsively 
employed after the learning experience. 


A bank of items and scales has been adopted 
or adapted by NCIC-TP to assess various 
aspects of implementation support 
performance. These items and scales can be 
used for discrete training events, support 
sessions, or site visits. Alternatively, they can 
be used as summative indicators’ of 
performance across time intervals or entire 
support periods. 


(1) Quality item: measures participants’ 
attitudes about the quality of training or 
support delivered. 


Members of 
external 
implementation 
support teams 
and support 
recipients 
collaboratively 
complete these 
forms 


Training 
participants and 
support 
recipients, 
usually coalition 
leaders and 
implementation 
team members 


Supported 
Performance 


Capacity 
Development 
Supported 
Performance; 


In development by 
NCIC-TP 


In development by 


; NCIC-TP 


At 


the end of the 


external 


implementation 
support period 
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(2) Use items: three items are available to 
measure participants’ beliefs about 
their future use of implementation 
strategies covered during training or 
support events (i.e., likely use, 
confidence to use, adequacy of support 
to put to use). Separately, an item is 
available to measure participants’ 
actual use of implementation strategies 
covered during historical training or 
support events. 


(3 


~— 


Usefulness item: measures support 
recipients’ retrospective beliefs about 
the usefulness of training or support 
that was received. 


(4 


~— 


Feasibility Scale: a five-item measure of 
implementation strategy feasibility, 
adapted from Weiner, Dorsey, Stanick, 
Halko, Powell, & Lewis.134 


(5) Appropriateness Scale: a_ five-item 
measure of training or support 
appropriateness, adapted from Weiner, 
Dorsey, Stanick, Halko, Powell, & 
Lewis.13? 


(6) Acceptability Scale: a five-item measure 
of training or support acceptability, 
adapted from Weiner, Dorsey, Stanick, 
Halko, Powell, & Lewis.134 
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